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frameworks, and scoring guides.) (Author/RS) 



* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 




C;ii ^07 6 To ED 420 895 




NATIONAL CENTER FOR EDUCATION STATISTICS 




NAEP and Theatre: Framework, Field Test, and Assessment 



Abstract: NAEP has developed its first set of assessment tasks 
for measuring student achievement in theatre. These tasks 
draw on a wide range of dramatic forms and use both paper- 
and-pencil and performance tasks. 

In 1992, the Arts Education Consensus Project, spon- 
sored by the National Assessment Governing Board 
(NAGB), began an 18-month effort to establish objec- 
tives for assessing arts instruction in the nation’s ele- 
mentary and secondary schools. The National Assess- 
ment of Educational Progress (NAEP) had assessed mu- 
sic education in 1972 and 1978 and visual arts education 
in 1975 and 1978. The National Endowment for the 
Arts, in collaboration with the Getty Center for Educa- 
tion in the Arts, funded the Arts Education Consensus 
Project, enabling it to create the NAEP Arts Education 
Assessment Framework. 

The Consensus Project 1 identified objectives for arts 
education in general and individually for dance, music, 
theatre, and the visual arts, in the fourth, eighth, and 
twelfth grades. It also developed Assessment Specifica- 
tions for each of the four arts, setting forth requirements 
for the content of the assessments in more detail. NAEP 
field-tested the theatre assessment for the fourth and 
eighth grades in 1995 and for the twelfth grade in 1997. 
In 1998, NAEP will publish a Field Test Process Report, 
on the development, administration, and scoring of arts 
tasks for the three grades. The report will cover all four 
arts, including theatre. 

In 1997, NAEP conducted an assessment of students’ 
achievement in the arts, for the eighth grade only. NAEP 
assessed students in music and the visual arts using a 
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nationally representative sample of all students, regard- 
less of their background in music or the visual arts. The 
theatre assessment was a targeted assessment, limited to 
students who would have completed at least 30 class 
hours in theatre instruction by the end of the 1996-97 
school year. The students attended schools providing at 
least 45 semester hours of classroom instruction in 
theatre for eighth graders. The sample for the theatre 
assessment was about 2,000 students, in both public and 
private schools, enough to obtain statistically valid re- 
sults. NAEP did not conduct a dance assessment, be- 
cause there were too few dance programs in schools to 




Theatre is rooted in the universal human impulse to play 
and to imitate. The NAEP Theatre Field Test and Assess- 
ment provide innovative methods for assessing student 
achievement in theatre. 

NOTE: For reasons of confidentiality, the students shown in this 
picture are not NAEP participants. 

Photo ot 1990 production ot “The Children's Hour courtesy Living Library 
Theatre, Inc., Santa Monica, California 
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Table 1. — NAEP Guidelines for Theatre Assessment 

Theatre assessment shall 

• Affirm theatre as a way of knowing with a unique capacity to 
integrate the intellect, emotions, and physical skills. 

• Honor theatre as a discrete art form, but also encourage stu- 
dents to see the artistic experience as a unified whole. 

• Examine and report on developing abilities of students. 

• Connect with students’ real-life experiences of theatre. 

• Evaluate students primarily through performance. 

• Go beyond quantification to include critical judgment. 

• Use background variables to recognize differences and ineq- 
uities in school resources (large-scale assessments only). 



• Address both the processes and products of the theatre, and 
expand the public’s awareness of the importance of each. 

• Employ a comprehensive vision of theatre education, in- 
cluding what ought to be in theatre education, rather than 
simply what is. 

• Produce information useful to a variety of audi- 
ences — students, parents, teachers, administrators, policy 
makers, artists, and other community members. 

• Sample student performance separately for general and spe- 
cialized theatre programs (large-scale assessments only). 

• Reflect a pluralistic view of theatre education, both in terms 
of individual products and the cultural bases of the theatre. 



Source: NAEP 1997 Arts Education Assessment Framework, National Assessment Governing Board 



allow for an adequate sample. Results of all three as- 
sessments, together with samples of arts tasks for all 
four arts, will' appear in the 1997 Arts Education Report 
Card, which will be published in the fall of 1998. 

The next two sections of this Focus on NAEP summa- 
rize the framework for the theatre assessment, which can 
be found in full in the NAEP 1997 Arts Education As- 
sessment Framework . Following these sections will be a 
discussion of the theatre field tests. 

The Importance of Theatre 

Theatre is rooted in the universal human impulse to 
play, imitate, create, and share ideas and feelings. A so- 
cial art form, theatre reveals both the human condition 
and the human experience. In both its traditional forms 
and the newer media of television and film, theatre 
transports players and audience through time and space 
without leaving the here and now. 

The NAEP Arts Education Assessment Framework is 
founded on a vision of a society that believes the arts, 
including theatre, are essential for every child’s com- 
plete development and education. To be fully educated, 
students must be able to draw on artistic experiences and 
knowledge as a means of understanding both themselves 
and the world around them. The expectation is not that 
all students will become professional artists, although 
some will. What is expected is that they will have expe- 
rienced enough of the discipline, the challenge, the joy 
of creating in different art forms to intimately under- 
stand the human significance of theatre, dance, music, 
and the visual arts (see table 1 for guidelines for theatre 
assessment). 

O 




The Theatre Assessment 
Framework: Content and 
Processes 

The theatre assessment covers both content and proc- 
esses. Content includes (1) knowledge and understand- 
ing of theatre and (2) perceptual, technical, expressive, 
and intellectual/reflective skills. Processes include (1) 
creating/performing, and (2) responding. While much of 
what students know, as well as what they can do, will be 
demonstrated through performance, the theatre assess- 
ment framework includes a written portion as well. Ta- 
ble 2 sets forth the theatre assessment framework in 
brief. 

Content 

Knowledge means knowing about the art of theatre; its 
history, cultural, social, and personal contexts; its forms 
and structure, how it creates meaning, and its aesthetic 
qualities. Students understand the literary, visual, aural, 
oral, kinesthetic, and psychological aspects of a theatri- 
cal event. They are able to engage in self-criticism, tak- 
ing into consideration form and structure, contexts, and 
aesthetic responses. 

Skills are the abilities associated with the technical, per- 
ceptual, and expressive processes of theatre. Included in 
this category are such activities as skills required in cre- 
ating a text, acting, staging, designing, and articulating a 
response. Abilities to create, perform, and respond in the 
theatre are predicated on the application of both knowl- 
edge and skills simultaneously. 
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Table 2. — Theatre Assessment Framework 


Processes 


Creating/Performing 


Responding 


When creating and performing in theatre, students: 

• develop scripts and scenarios; 

• make design and technical choices to communicate locale and 
mood for dramatic materials for theatre, film, and television; 

• develop characters through an acting process; and 

• direct by interpreting dramatic texts and organizing time and 
people in planning and rehearsing improvised and scripted 
scenes. 


When perceiving, analyzing, interpreting, and judging works in 
theatre, students: 

• describe and analyze artistic choices in their own work and 
construct meaning; 

• describe and compare elements, styles, genre, media, and 
dramatic literature; 

• place work in context (personal, social, historical, and cul- 
tural); and 

• evaluate performances as audience and critic. 


Content 


Knowledge 


Skills 


Applying knowledge of: 
Context 

personal 
social 
cultural 
historical 
Aesthetics 
Form and structure 
Processes 


Applying cognitive, affective, and motor skills, including: 
Perceptual 

Intellectual/Reflective 

Expressive 

Technical 


SOURCE: NAEP 1997 Arts Education Assessment Framework, National Assessment Governing Board 



Processes 

Creating and Performing are closely linked in theatre. 

At the same time, different aspects of theatre call for 
different creative processes. All call for imagining, con- 
ceiving, and generating. Playwriting requires the use of 
all three processes to create character, story, dramatic 
actions, and dialogue. Acting involves discovering and 
developing emotions and creating a series of actions for 
a character. Designing requires discovering, developing, 
and organizing an environment. In order to direct, stu- 
dents must analyze a script and develop an interpretation 
by organizing the time, place, spaces and rhythms of a 
production. Creating in film making and television de- 
mand the development and organization of the elements 
of theatre as well as the special elements of the medium. 

Performing is central to theatre studies. Performing is 
the evidence of creating; it is the process viewed and 
heard by an audience. Performing is a highly complex 
collaborative activity in which the performer is aware of 
the audience and fellow performers, responding and ad- 
justing the performance accordingly. The ways of 
learning demanded by performance are equally complex. 



requiring the interplay of all the processes noted in cre- 
ating. Performing is impossible to assess through any 
other means than through the processes themselves. 

Responding includes both students' reactions as specta- 
tors to each other's work and reflecting on their own 
work. They respond to outside artists and performers. 
They examine artistic choices in ideas, stories, scripts, 
designs, and actions by explaining, describing, clarify- 
ing, comparing, and evaluating. They apply their knowl- 
edge of the theatre and analytical skills to determine 
which elements are successful in a performance; but 
responding in theatre is more than analysis. Responding 
involves emotional and intuitive behaviors as well. Stu- 
dents become responsive audience members of theatrical 
performances, films, and tele-play presentations. They 
recognize a variety of theatrical genres and styles and 
identify and compare them in theatre, film, and televi- 
sion. 

The Field Test Samples 

The field test of the NAEP theatre assessment tasks for 
grades four and eight were conducted in 1995, while the 
twelfth-grade tasks were tested in 1997. A general 
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Table 3. — Guidelines for Stimulus Materials for Theatre Tasks 




Grade 4 


Grade 8 


Grade 12 


Theatre 


10% 

• Dramatic Literature 

• Children’s Plays 


30% 

• Rituals (Non-European Cultures) 

• Shakespeare 

• Comedy 

• American Musical 

• Serious Drama 


40% 

Draw from Grade 8 categories and add: 

• Epic Theatre 

• Spanish Golden Age 

• Greek Tragedy 

• 20th Century American and World Drama 


Literature 


40% 

• Fairy Tales 

• Folk Tales 

• Children’s Literature 

• Poetry 


25% 

Draw from Grade 4 categories and add: 

• 1 9th Century American/ European 

• Modern 

• Non-Western 


15% 

Draw from Grade 8 categories 


Film and 
Television 


10% 

• Television 

• Film 


20% 

Draw from Grade 4 categories and add: 

• Social Realism 

• Special Effects 


25% 

Draw from Grade 8 categories and add: 

• Documentary 

• Experimental 

• Foreign Films 


Other 


40% 

• Other Art Forms 

• Historical and Cur- 
rent Events 


25% 

Draw from Grade 4 categories and add: 
• Anthropology 


20% 

Draw from Grade 8 categories 



population of students participated in the field tests for 
grades four and eight. However, in the twelfth-grade 
field test, only students who had participated in in- 
school theatre classes were included in the sample. 
About 2,000 students participated in each of the field 
tests. The twelfth-grade field tests were held to evaluate 
the validity of the theatre tasks, even though NAEP had 
determined that it would not conduct an assessment for 
twelfth-grade students. 

Theatre Tasks 

NAEP used two types of assessment tasks in the field 
tests: paper-and-pencil tasks and performance tasks. As- 
sessment tasks were prepared by the Educational Test- 
ing Service, under the guidance of a committee of thea- 
tre education experts. The fourth-grade field test de- 
voted about 60 minutes to the paper-and-pencil tasks and 
15-30 minutes to the performance task. The eighth- 
grade field test devoted 60 and 30 minutes, respectively, 
while the twelfth-grade field test devoted 60 minutes 
and 60-90 minutes, respectively. In all cases, assess- 
ment times are approximate only, depending on the spe- 
cific tasks involved. 

The performance tasks assessed the creating and per- 
forming aspects of theatre. The paper-and-pencil tasks 
assessed students’ abilities to respond to theatre, but 
also required them to make and defend a wide variety of 
creative decisions as well. To ensure coverage of all as- 
O 
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pects of theatrical knowledge, and to lighten the burden 
on a given student, NAEP developed a number of differ- 
ent task “blocks,” for both the paper-and-pencil and per- 
formance tasks. Students in the sample were assigned at 
random to several paper-and-pencil blocks and one per- 
formance block. 

NAEP performance tasks are designed to be as authentic 
as time and resources permit. However, authenticity is 
limited by the fact that the assessment tasks must also 
offer the same opportunities and the same challenges, in 
the same circumstances, for all the students assessed. No 
comparison across students is possible without such 
standardization. NAEP plans to make available to the 
public all tasks, both paper-and-pencil and performance, 
that are not covered by copyright. 

1. Paper-and-Pencil Tasks 

In addition to having students respond to theatrical per- 
formances, paper-and-pencil tasks asked students to ad- 
dress such issues as set design, decor, costume, casting, 
cinematic techniques, sound effects, and direction. (Ex- 
amples are drawn throughout from the twelfth-grade 
field test.) 

Students read excerpts from plays, looked at photo- 
graphs, posters, and other visual stimuli, and saw video 
clips, both from films and from a tape of a live theatrical 
presentation. (See table 3 for a breakdown of the kinds 
of materials used in the field tests, taken from the Con- 
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sensus Project’s Assessment Specifications.) The tasks 
used multiple-choice, short constructed-response, and 
extended constructed-response questions (see table 4 for 
a sample scoring guide for an extended constructed- 
response task). Over two thirds were constructed- 
response rather than multiple-choice. 

2. Performance Tasks 

Performance tasks asked students to work together in 
small groups (usually two to four students) to do such 
things as improvise a scene on the basis of a few intro- 
ductory lines of dialogue or to write and perform a brief 
play. Video cameras recorded student performances. In 
some of the tasks, students were asked to comment on 
their work — what they were trying to achieve, and how 
well they did it. 

Because of the collaborative nature of theatre, a stu- 
dent’s performance in several of the tasks was a combi- 
nation of an individual and a group score, unusual in 
NAEP. Both the paper-and-pencil and the performance 
tasks allowed students recognition for exceptional per- 
formance. Scoring guides for the performance tasks gave 
credit for students who performed their tasks in a “mas- 
terful” manner (see table 5 for a sample scoring guide 
for a performance task). 

Field Test Issues 

In the theatre field tests for grades four and eight there 
were many issues that NAEP had not dealt with before. 
Capturing student group performances (two to four stu- 
dents) on video for evaluation presented many chal- 
lenges. A workable space had to be prescribed on the 
floor. A video camera, which would remain stationary 
throughout the performance, had to be properly placed 
to record the entire scene. Backgrounds were not always 
taken into account, and there was no special lighting. 



Table 4. — Twelfth-Grade Theatre Field Test 
Scoring Guide for Paper-and-Pencil Task 

Extended constructed-response questions on the paper-and- 
pencil test gave students such tasks as determining whether 
to use a realistic or abstract set for a play. The scoring 
guide, designed to capture the full range of student abilities, 
gave credit according to standards similar to the following: 

Unacceptable: The student makes no choice between a 
realistic or abstract set; the student makes a choice without 
explaining it; the student’s explanation is unclear. 

Partial: The student makes a choice, but cites only one 
reason for that choice, and offers no basis for the choice 
from the text of the play. 

Essential: The student makes a choice and gives two rea- 
sons for that choice, but fails to provide support from the 
text of the play or provides support that is unclear or vague. 

Extensive: The student makes a choice and gives two rea- 
sons for that choice, giving clear support from the text of 
the play. In addition, the response indicates an understand- 
ing of the difference between an abstract and a realistic set. 



which meant that it was sometimes difficult to see facial 
expressions. Since the microphone was on the camera, it 
was sometimes difficult to hear actors' voices, particu- 
larly if the task took place in a noisy area. Facilitators 
had to have special training, so that they could make the 
students feel comfortable enough to perform without 
being “too helpful” — unfairly contributing to the stu- 
dents' performance. 

In the twelfth-grade field test, NAEP found it necessary 
to increase the amount of time allowed for performance 
tasks from 60 to as much as 90 minutes. Traditionally, 
NAEP has sought to use no more than one class period 
for an assessment task. However, NAEP found that 
some of the theatre performance tasks required more 
time, particularly the tasks that required students to cre- 
ate, perform, and then analyze their work. 



Table 5. — Twelfth-Grade Theatre Field Test Scoring Guide for a Performance Task 



Student performance tasks were scored using such factors as the 
degree of spatial awareness demonstrated by students in their per- 
formance. Scoring guides used criteria similar to the following: 

Unacceptable: The student demonstrates no sense of spatial 
awareness, with regard to herself/himself, the other actors, or the 
audience. 

Partial: The student demonstrates a spatial awareness some of 
the time. However, at some points the student is either blocked by 
or blocks another actor and/or speaks with her/his back to the 
audience. 

O 



Essential: The student demonstrates a spatial awareness a major- 
ity of the time. The student may have one minor lapse by either 
blocking or being blocked by another actor and/or speaking with 
her/his back to the audience. 

Extensive: The student demonstrates consistent spatial awareness 
during the entire improvised scene, with no lapses of any kind. 
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The greatest difficulty encountered in the field tests was 
obtaining a full range of performance from the students 
sampled. This was particularly true in the eighth-grade 
field test. In the fourth- and eighth-grade field tests, 
which were given to random cross sections of students, 
it was difficult to obtain performances that went beyond 
the simplest level. To fairly assess the students and ob- 
tain results that resembled the breadth of the theatre 
standards, NAEP decided to sample only students who 
had had some instruction in theatre, for both the twelfth- 
grade field test and the eighth-grade assessment. The 
difference was significant. Student performance evi- 
denced the full range envisioned by the scoring rubrics. 

Creating and administering a national performance as- 
sessment was very challenging. The NAEP Theatre 
Framework, Field Test and Assessment provide useful 
models for future assessments, at the state and local 
level, as well as nationally. Several important problems 
were confronted during the development and scoring of 
the assessment. Among these were how to create tasks 
suitable for a variety of students with a variety of edu- 
cational backgrounds, while also making distinctions 
among levels of student achievement. These issues, 
among others, will be explored in the upcoming arts re- 
ports. 

Footnotes 

1 . For details on the Arts Education Consensus Project, see the 
Arts Education Assessment Framework for the 1997 National 
Assessment of Educational Progress (1994), the National As- 
sessment Governing Board. 

For Further Information 

Arts Education Assessment Framework for the 1997 National 
Assessment of Educational Progress. Single copies are avail- 
able free from the National Assessment Governing Board, 800 
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North Capitol Street, NW, Suite 825, Washington, DC 
20002-4233. Copies may also be obtained on the World Wide 

Web at http://www.nagb.org/pub.html. 

National Standards for Arts Education. Copies are available 
for $20 from the Music Educators National Conference, 1806 
Robert Fulton Drive, Reston VA 20191, 800-336-3768. 

Arts Education in Public Elementary and Secondary Schools, 
NCES 95-082, provides data obtained from questionnaires 
sent to the principals of nationally representative samples of 
public schools. It is not in print, but is available over the 
NAEP web site at http://nces.ed.gov/NAEP/. 

The following Focus on NAEP publications are also available 
free from the National Center for Education Statistics, U.S. 
Department of Education, Washington, DC 20208-5653. 
Copies may also be accessed over the World Wide Web at 
http://nces.ed.gov/NAEP/. 

The NAEP Arts Education Framework , Field Test , and As- 
sessment, NCES 98-527 

NAEP and Dance: Framework , Field Test , and Survey, NCES 
98-459 

NAEP and Music: Framework , Field Test , and Assessment, 
NCES 98-529 

NAEP and the Visual Arts: Framework , Field Test , and As- 
sessment, NCES 98-526 

The Focus on NAEP series briefly summarizes information on 
the ongoing development and implementation of the National 
Assessment of Educational Progress (NAEP). The series is a 
product of the National Center for Education Statistics 
(NCES), Pascal D. Forgione, Jr., Commissioner, and Gary W. 
Phillips, Associate Commissioner for Education Assessment. 
This Focus on NAEP was written by Alan Vanneman, of the 
Education Statistics Services Institute, in support of NCES, 
Claudette Morton, chair of the NAEP Arts Standing Com- 
mittee and co-chair of the Theatre Planning Committee, and 
Lisa Beth Allen of the Sundance Institute, Theatre Program. 
To order other NAEP publications, call Bob Clemons at 202- 
219-1690, or e-mail bob_clemons@ed.gov. 

The NCES World Wide Web Home Page is 
http://nces.ed.gov/. 
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